refactor(computer-use): phase-0 quick wins from computer-use/browser-control audit - #1777

…on safe

An end-to-end review of the consolidated browser path (walking a real
task: open -> log in -> find -> click -> observe -> read) found the
advertised workflow could not actually be executed by a model.

Results the model never saw:
- The tool pipeline prefers `result_for_assistant`, and every browser
  action passed only a one-line count there, so `snapshot` returned
  literally "Snapshot: 42 interactive elements" and the @en refs the
  documented connect -> snapshot -> click flow depends on never reached
  the model. Same for list_pages/tab_query (no page ids, so switch_page
  was unusable), get_html, cookies, network/console/errors and
  meta.capabilities. Each now returns the payload the next decision
  needs; screenshot returns a real image attachment instead of a byte
  count and no longer duplicates base64 in the data field.

Actions that silently did the wrong thing:
- snapshot renumbered @en from 1 without clearing the previous pass's
  data-cdp-ref attributes, so after scrolling (viewport-filtered
  numbering) a stale attribute earlier in document order won the
  querySelector and click acted on the wrong element while reporting
  success. Refs are now cleared across the document, open shadow roots
  and same-origin frames before renumbering.
- click/hover dispatched mouse events at a point without hit-testing it;
  an occluded target reported success. The center point is now verified
  with elementFromPoint and a covered target is refused with a
  recoverable error.
- press_key sent no text/windowsVirtualKeyCode, so Enter never triggered
  native form submission.
- browser.fetch wrapped CORS/transport failures in a "Fetched <url>"
  success line; they are now structured errors that explain the
  same-origin constraint, and HTTP status is surfaced in the summary
  while the response body is preserved.

Frontmost-app detection (regression from the guard activated in GCWing#1777):
- On Windows the foreground "name" is the window *title*, and the
  Chromium check matched bare substrings, so any window titled "Search"
  matched "arc" and had all desktop input blocked. Foreground apps now
  carry a process_name (executable basename on Windows, process name on
  macOS) and matching is whole-identity, falling back to token matching
  on the display name only when no identity is available.
- The guard covers the app_*/interactive_*/visual_* variants that could
  previously bypass it, exempts the alt+tab/command+tab focus switch so
  a non-browser task can escape, and short-circuits on explicit
  non-browser selectors instead of querying the host.

Also: select/wait accept @en refs, stale refs report STALE_REF instead
of a generic not-found, unresolvable cross-origin iframes are explained
instead of retried forever, prompts no longer promise the user's login
state (connect uses BitFun's managed profile), open_builtin is routed as
the display-only path, prompts stop pointing unconditionally at the
default-off ComputerUse tool, and the agent-browser skill no longer
argues against itself when explicitly invoked.

…th (#1779)

* refactor(browser-control): consolidate to a single default browser path

The audit found seven overlapping browser-control paths; the model
oscillating between two full automation stacks (agent-browser CLI vs
ControlHub CDP, each with its own @en ref namespace, browser instance
and login state) was the largest source of failed browser operations.

- Make agent-browser default-off in every mode so the ControlHub
  browser domain is the single default path; explicit /skill invocation
  and user overrides still opt in (extend the default-hidden whitelist
  so explicit invocation keeps working)
- Fix iframe click coordinates: element_center now accumulates ancestor
  frame offsets along the frameElement chain; cross-origin frames
  surface a structured error advising snapshot + re-target; missing
  coords error instead of clicking (0,0)
- Remove the dead frame/frame_main actions and the unread active_frame
  session state they wrote; they never affected execution context
- Emit structured error codes with recovery instructions at the source
  (element not found -> NOT_FOUND "take a new snapshot", dead session ->
  WRONG_TAB "reconnect or switch_page", timeouts -> TIMEOUT) instead of
  relying on phrase-matching classification; phrase matching stays as
  fallback only
- State non-observability where control leaves the CDP stack:
  open_builtin marks the panel display-only, open_url/open_file explain
  the opened window cannot be observed or controlled and route to the
  right tool, WebFetch description routes login/JS pages to the browser
  domain
- Record the product decision that ai.computer_use_enabled and browser
  control are independent switches (browser control stays available
  when computer use is off) in the refactor plan

* fix(browser-control): make browser results actionable and app detection safe

An end-to-end review of the consolidated browser path (walking a real
task: open -> log in -> find -> click -> observe -> read) found the
advertised workflow could not actually be executed by a model.

Results the model never saw:
- The tool pipeline prefers `result_for_assistant`, and every browser
  action passed only a one-line count there, so `snapshot` returned
  literally "Snapshot: 42 interactive elements" and the @en refs the
  documented connect -> snapshot -> click flow depends on never reached
  the model. Same for list_pages/tab_query (no page ids, so switch_page
  was unusable), get_html, cookies, network/console/errors and
  meta.capabilities. Each now returns the payload the next decision
  needs; screenshot returns a real image attachment instead of a byte
  count and no longer duplicates base64 in the data field.

Actions that silently did the wrong thing:
- snapshot renumbered @en from 1 without clearing the previous pass's
  data-cdp-ref attributes, so after scrolling (viewport-filtered
  numbering) a stale attribute earlier in document order won the
  querySelector and click acted on the wrong element while reporting
  success. Refs are now cleared across the document, open shadow roots
  and same-origin frames before renumbering.
- click/hover dispatched mouse events at a point without hit-testing it;
  an occluded target reported success. The center point is now verified
  with elementFromPoint and a covered target is refused with a
  recoverable error.
- press_key sent no text/windowsVirtualKeyCode, so Enter never triggered
  native form submission.
- browser.fetch wrapped CORS/transport failures in a "Fetched <url>"
  success line; they are now structured errors that explain the
  same-origin constraint, and HTTP status is surfaced in the summary
  while the response body is preserved.

Frontmost-app detection (regression from the guard activated in #1777):
- On Windows the foreground "name" is the window *title*, and the
  Chromium check matched bare substrings, so any window titled "Search"
  matched "arc" and had all desktop input blocked. Foreground apps now
  carry a process_name (executable basename on Windows, process name on
  macOS) and matching is whole-identity, falling back to token matching
  on the display name only when no identity is available.
- The guard covers the app_*/interactive_*/visual_* variants that could
  previously bypass it, exempts the alt+tab/command+tab focus switch so
  a non-browser task can escape, and short-circuits on explicit
  non-browser selectors instead of querying the host.

Also: select/wait accept @en refs, stale refs report STALE_REF instead
of a generic not-found, unresolvable cross-origin iframes are explained
instead of retried forever, prompts no longer promise the user's login
state (connect uses BitFun's managed profile), open_builtin is routed as
the display-only path, prompts stop pointing unconditionally at the
default-off ComputerUse tool, and the agent-browser skill no longer
argues against itself when explicitly invoked.

* fix(browser-control): pierce shadow roots in the click hit-test

The occlusion hit-test compared document.elementFromPoint against the
target with contains(), but elementFromPoint stops at a shadow host and
host.contains(shadowChild) is false — so every element inside an open
shadow root (which resolve/snapshot deliberately support) was reported
as occluded by its own host and refused. Descend through open shadow
roots' elementFromPoint before comparing.

---------

Co-authored-by: bowen628 <bowen628@noreply.gitcode.com>

…th (GCWing#1779)

* refactor(browser-control): consolidate to a single default browser path

The audit found seven overlapping browser-control paths; the model
oscillating between two full automation stacks (agent-browser CLI vs
ControlHub CDP, each with its own @en ref namespace, browser instance
and login state) was the largest source of failed browser operations.

- Make agent-browser default-off in every mode so the ControlHub
  browser domain is the single default path; explicit /skill invocation
  and user overrides still opt in (extend the default-hidden whitelist
  so explicit invocation keeps working)
- Fix iframe click coordinates: element_center now accumulates ancestor
  frame offsets along the frameElement chain; cross-origin frames
  surface a structured error advising snapshot + re-target; missing
  coords error instead of clicking (0,0)
- Remove the dead frame/frame_main actions and the unread active_frame
  session state they wrote; they never affected execution context
- Emit structured error codes with recovery instructions at the source
  (element not found -> NOT_FOUND "take a new snapshot", dead session ->
  WRONG_TAB "reconnect or switch_page", timeouts -> TIMEOUT) instead of
  relying on phrase-matching classification; phrase matching stays as
  fallback only
- State non-observability where control leaves the CDP stack:
  open_builtin marks the panel display-only, open_url/open_file explain
  the opened window cannot be observed or controlled and route to the
  right tool, WebFetch description routes login/JS pages to the browser
  domain
- Record the product decision that ai.computer_use_enabled and browser
  control are independent switches (browser control stays available
  when computer use is off) in the refactor plan

* fix(browser-control): make browser results actionable and app detection safe

An end-to-end review of the consolidated browser path (walking a real
task: open -> log in -> find -> click -> observe -> read) found the
advertised workflow could not actually be executed by a model.

Results the model never saw:
- The tool pipeline prefers `result_for_assistant`, and every browser
  action passed only a one-line count there, so `snapshot` returned
  literally "Snapshot: 42 interactive elements" and the @en refs the
  documented connect -> snapshot -> click flow depends on never reached
  the model. Same for list_pages/tab_query (no page ids, so switch_page
  was unusable), get_html, cookies, network/console/errors and
  meta.capabilities. Each now returns the payload the next decision
  needs; screenshot returns a real image attachment instead of a byte
  count and no longer duplicates base64 in the data field.

Actions that silently did the wrong thing:
- snapshot renumbered @en from 1 without clearing the previous pass's
  data-cdp-ref attributes, so after scrolling (viewport-filtered
  numbering) a stale attribute earlier in document order won the
  querySelector and click acted on the wrong element while reporting
  success. Refs are now cleared across the document, open shadow roots
  and same-origin frames before renumbering.
- click/hover dispatched mouse events at a point without hit-testing it;
  an occluded target reported success. The center point is now verified
  with elementFromPoint and a covered target is refused with a
  recoverable error.
- press_key sent no text/windowsVirtualKeyCode, so Enter never triggered
  native form submission.
- browser.fetch wrapped CORS/transport failures in a "Fetched <url>"
  success line; they are now structured errors that explain the
  same-origin constraint, and HTTP status is surfaced in the summary
  while the response body is preserved.

Frontmost-app detection (regression from the guard activated in GCWing#1777):
- On Windows the foreground "name" is the window *title*, and the
  Chromium check matched bare substrings, so any window titled "Search"
  matched "arc" and had all desktop input blocked. Foreground apps now
  carry a process_name (executable basename on Windows, process name on
  macOS) and matching is whole-identity, falling back to token matching
  on the display name only when no identity is available.
- The guard covers the app_*/interactive_*/visual_* variants that could
  previously bypass it, exempts the alt+tab/command+tab focus switch so
  a non-browser task can escape, and short-circuits on explicit
  non-browser selectors instead of querying the host.

Also: select/wait accept @en refs, stale refs report STALE_REF instead
of a generic not-found, unresolvable cross-origin iframes are explained
instead of retried forever, prompts no longer promise the user's login
state (connect uses BitFun's managed profile), open_builtin is routed as
the display-only path, prompts stop pointing unconditionally at the
default-off ComputerUse tool, and the agent-browser skill no longer
argues against itself when explicitly invoked.

* fix(browser-control): pierce shadow roots in the click hit-test

The occlusion hit-test compared document.elementFromPoint against the
target with contains(), but elementFromPoint stops at a shadow host and
host.contains(shadowChild) is false — so every element inside an open
shadow root (which resolve/snapshot deliberately support) was reported
as occluded by its own host and refused. Descend through open shadow
roots' elementFromPoint before comparing.

---------

Co-authored-by: bowen628 <bowen628@noreply.gitcode.com>